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ABSTRACT 


The Air Force Weather Agency (AFWA) uses the World Wide Merge 
Cloud Analysis (WWMCA) to display cloud amounts onto a hemispheric 
stereographic projection map. The goal of this study was to verify the WWMCA 
against real-time surface weather observations in the same spatial and temporal 
scale. The utilization of MapServer, a Geographic Information System (GIS) tool, 
to make these comparisons was essential in this study. The comparisons were 
10 different Air Force bases across the continent of the United States for 16 
days. Discrepancies existed between the drier climate and fair climate regions 
as compared to more active weather regions. Nellis and Travis AFB had higher 
number of verified observation as compared to the other eight bases. Maxwell 
AFB had the highest percentage of poorly verifying observations with 44% from 
the observer only results. Overall, the WWMCA did not verify well with a 
verification of 27% and a miss rate of 32%. Therefore, the Air Force Weather 
Agency (AFWA) needs to look at further improving cloud model output. This 
study shows some of the shortcomings of WWMCA cloud model data and the 
potential benefits to AFWA if improvements are made to cloud model output. 
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I. INTRODUCTION 


A. OVERVIEW 

The focus of this research is to compare the World Wide Merge Cloud 
Analyses (WWMCA) analyses with satellite and real time observational data. 
Correctly initializing the data between real time and cloud model data is 
imperative to produce a better cloud model. In order to achieve a better 
comparison between model data and real time data a Geographic Information 
System (GIS) tool will be used to make these evaluations. The benefit in using 
the GIS tool is that it helps to resolve problems with spatial and temporal scales, 
which are normally associated with a grid model system. Improvements in the 
cloud models will benefit military planners and operators who depend on 
accurate cloud forecasts to plan critical missions based on meteorological 
forecasts. 

B. PROBLEM 

There is an inherent difficulty with verifying cloud models because of the 
spatial and temporal scale differences from observed data. Synoptic 
observations detail what a certified meteorologist views from the ground. If low 
clouds obscure the sky, then the observer will report the obscured layer. The 
observer cannot see any cloud layers above this layer. This can present a 
problem in verifying the cloud layer if there were clouds. This research will look 
at this problem with verifying cloud models by taking the raw cloud data and 
viewing it from a geospatial perspective. This will allow for a more accurate 
verification method of the clouds. 

C. SCOPE 

The scope of this thesis is to resolve the problem with verifying cloud 
model data. In order to understand how to verify the cloud model with a surface 
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weather observation there has to be an understanding on how the WMCA data is 
produced. This is helpful because it shows the process in producing the 
WWMCA. This key insight into this process will be helpful in the development of 
a verification method. Some of methods of verifying model data include 
comparing it to satellite images, surface weather observations, or rawindsond 
data. This thesis will not produce a new algorithm for the Cloud Depiction and 
Forecast System (CDFS) II, which produces the WWMCA. However, this study 
will show the customer useful tools and techniques for improving the cloud model 
and potentially improving the algorithm. 

D. METHODOLOGY 

In order to do verify the cloud model, a systematic approach has to be 
taken to accurately verify the model. The approach to verify the WWMCA cloud 
model consists of: 

1. Gather raw weather data into a database for storage. Data will be 
screened to make sure there are no errors or missing information. 

2. Compare the data using GIS tools to help ensure an accurate 
comparison. 

3. Analyze the results from the comparison. This will include all of the 
test results from this research. 

4. Interpret the results to understand if there are any biases or 
difference in the model from the observed data. 

5. Display the results in a chart or diagram to provide an overview of 
the findings from this research. The charts or diagram will display any trends or 
anomalies in the data. 
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E. 


ORGANIZATION OF THESIS 


Chapter II consists of a literature review of the nephanalysis, and Cloud 
Depiction Forecast System II (CDFS II) used at the Air Force Weather Agency 
(AFWA). This chapter will also discuss current verification methods, and the 
history of cloud models. Data and methods will be shown in Chapter III. This is 
important in describing how the data will be gathered and stored. It will also 
show how the comparison will be made between the cloud model, and the 
synoptic weather observations. Chapter IV will present the results of the 
verification, and trends from these results. Chapter V will show the conclusion 
and possible future works. 
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II. BACKGROUND 


A. REAL TIME NEPHANALYSIS (RTNEPH) 

1. History 

In 1983, Air Force Global Weather Central’s (AFGWC), now AFWA, 3- 
Dimensional NEPHanalysis (3DNEPH) was replaced by RTNEPH. The 
RTNEPH cloud model combined satellite data with conventional data (Fye, 
1978). The merger of the two data allowed for an automated cloud analysis at 25 
nm horizontal resolution (Kiess, 1988). RTNEPH and 3DNEPH are similar but 
are written in different programming language. Consolidation allowed for easy 
maintenance of programming code to the RTNEPH model. The newer code also 
allowed for improvements to algorithms in the RTNEPH. The differences 
between the two models are the database and diagnostic information (Kiess, 
1988). An additional, significant improvement in 3DNEPH came with the 
upgrade from 15 fixed layers to four floating vertical layers (Kiess, 1988). 
Diagnosing and storing the bottom and top of the clouds improved vertical 
resolution compared to 3DNEPH. Also, the diagnostic definitions made for better 
control of inaccurate information in the database (Kiess, 1988). 

2. Model 


a. Problems with Models 

It is difficult to understand the atmosphere and quantify it with 
mathematical equations; Nebeker (1995) provides an excellent history. The 
equations that do describe the atmosphere, in a certain state, are generally 
approximated; therefore, there is some uncertainty in the equations. Another 
problem with computer models is that the surface weather observations that are 
incorporated into the model are not generally spread over entire areas of land. 
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Observations are sparse over uninhabited or underdeveloped areas. These 
problems contribute to the difficulty in modeling the atmosphere (Conklin, 1992). 

b. Algorithm and Initial Conditions in the Models 

The fundamental problems facing numerical weather prediction are 
getting the physics, initial condition, and resolution modeled, and interacting 
correctly. Each of these areas contributes to the total problem when it comes to 
modeling numerical weather. The physics portion of the algorithm tries to deal 
with the actual weather theories and calculations. Sometimes the weather 
conditions have to be parameterized because the computer model cannot 
process the weather data. When these weather conditions are parameterized 
then the physics portion is not described accurately. This can lead to 
inaccuracies in the model, when there are many weather conditions which are 
parameterized. One weather component that models have trouble accounting for 
is turbulence. It is also important to get the initial condition correct. As 
mentioned above, it is critical to get this correct because of the model 
dependency on the observed data (Conklin, 1992). 

3. Horizontal and Vertical Grids 

RTNEPH models the global nephanalysis on a 25 nm polar stereographic 
grid. Each grid point has four cloud layers in the eighth mesh grid. The 
motivation for switching to this grid was because of customer requirements and 
the similarity in features it had with 3DNEPH (Kiess, 1988). 

a. Data 

The satellite ingest, as of 1988, came from the AFGWC Satellite 
Global Data Base (SGDB). The raw (unprocessed) is at 1.5 nm. RTNEPH 
processes the images at 3 nm resolution, giving gray shade output (Kiess, 1988). 
RTNEPH uses 8X8 array to combine these points to form the cloud layer. There 
must be enough points in this merger to form a viable cloud image. With too few 
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data points, it is difficult to resolve the cloud feature (Kiess, 1988). On the other 
hand, if there is an over saturation of array values then there is a loss in 
resolution from the raw satellite image. Observations are eventually combined 
with the new processed satellite image. The area for the surface observations 
and other meteorological data, which is eventually incorporated, covers 20 to 50 
nm from the cloud data. The resolution of the grid has to be correct because if 
the grid is coarse then some surface observations could be thrown away or 
merged incorrectly. Finer grids could change the satellite analysis used in 
merging the two data types, thus affecting the output (Kiess, 1988). 

b. Hardware 

There are restrictions with the size of the grids because in 1988 
memory storage was expensive. The grids could not be smaller than 25 nm 
because grid spacing and database memory were limited by state of the science. 
Also, it took a lot of memory usage to process the cloud model. This was the 
dilemma with running RTNEPH in the production cycle. The previous model 
used for cloud analysis was 3DNEPH. Eighth-mesh was chosen because it was 
easy to convert the data from the previous model to RTNEPH (Fye, 1978). 

4. Grid Diagram 

Figure 1 below shows an RTNEPH grid with a map of the northern 
hemisphere (polar-stereography). Each grid’s side represents 1600nm in length. 
Figure 2 shows the southern hemisphere (Kiess, 1988). 
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Figure 1 and Figure 2 in the above polar stereographic projection has two 
grids. The two grids are part of the AFGWC Whole-mesh Reference Grid, and it 
has a resolution of 25nm. The diagram below shows the eight-mesh grid at 
25nm resolution (Cantrell, 2002). The Whole-mesh Reference Grid is at 200nm 
resolution and has a true projection at 60°. There are a total of 262,144 points 
(Floke, 1981). A grid contains a 512 x 512 matrix of points, for RTNEPFI, with the 
poles located at grid point (257,257). The WWMCA has 1024 X 1024 matrix of 
points (Kiess, 1988). 
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Figure 3. Mesh division from a Neph-Box grid on a hemispheric polar 
stereographic projection down to 64 th mesh (6 km or resolution 
of a cloud pixel). RTNEPH analysis data is archived at 8 th 
mesh (48 km) resolution. (From: Cantrell, 2002) 


The grid is subdivided into a set of 64 RTNEPFI boxes arranged in an 8 x 
8 matrix, and numbered 1 to 64 (Figure 1 and Figure 2). Each box contains a 64 
x 64 set of eight-mesh points. If a point is off the map projection (beyond the 
equator), it is not processed. The four corner boxes, 1, 8, 57, and 64, are all 
completely off the map projection and are not included in the RTNEPFI database 
(Kiess, 1988). The vertical grid from the cloud model has four different layers. 
These layers have information on cloud types, cloud heights, total cloud amount, 
and the base height of the cloud. In the RTNEPFI model cloud layers can be 
arranged in any area. The RTNEPFI sorts the cloud layers differently from 
3DNEPFI. Layer 1 has the base, which is not fixed and is further from the 
bottom. The cloud base and upper boundary are not stationary. The model 
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restricts the type of data allowed which helps with better resolution. Height of a 
particular layer has a range from the ground to 21900m mean sea level (MSL). 
Resolution for spacing is every 300m above 6000m MSL and below it the 
spacing is 30m (Kiess, 1988). 

B. CLOUD DEPICTION AND FORECAST SYSTEM (CDFS) II 

1. Introduction 

Cloud Depiction and Forecast System (CDFS) II is a computer system 
which combines raw data into a World Wide Merge Cloud Analysis (WWMCA). 
Observations and raw satellite are merged in a two-fold process (HQ 
AFWA/DNXM 2005). The first process, takes place when raw satellite images 
arrive in CDFS II. The satellite images are processed as Level 1, 2, and 3, 
shown in Figure 4. In the second process the WWMCA is produced from the 
most current satellite image and weather observations. The WWMCA analyses 
are produced every hour and stored. If the data is modified, then it is stored in 
the Unclassified Central Database (UCDB), where it is saved for 48 hours. If the 
data is not modified, then it is stored in local storage. Whether stored locally, or 
in the UCDB, the current data is used to initialize the next WWMCA run. The 
WWMCA is preserved on AFWA 16 th mesh grid. If there are changes made to 
the Non-modified WWMCA, these changes are sent back to the Satellite Data 
Handling System (SDHS), and the composite product becomes the Modified 
WWMCA (HQ AFWA/DNXM, 2005). 
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Conventional 

Algorithm 


Analysis 


Worldwide Merged Analysis 


Figure 4. CDFS II Multi-Source Cloud Analysis and Integration 
Procedure (From: HQ AFWA/DNXM, 2005) 


2. Data Processing 

Three models are incorporated into processing and analyzing 
meteorological data (HQ AFWA/DNXM, 2005). The nephanalysis model is used 
in the cloud depiction analysis by CDFS II. Advect Cloud model (ADVCLD) is 
used for cloud forecasting. The final model incorporated into CDFS II is the 
Surface Temperature (SFCTMP) model. This model handles analysis and 
forecasts used in CDFS II. Within CDFS II, the four levels of data processing are 
level 1, ingest processing; level 2, cloud detection; level 3, mapping onto grid 
projection at 16 th mesh resolution; and level 4, merge processing. Table 1 shows 
the products that are produced by WWMCA (HQ AFWA/DNXM, 2005). 
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a. 


Level 1 


CDFS II receives data (Raw Data Records, RDR) from a variety of 
satellites (e.g., DMSP, AVHRR, and GOES). CDFS II specifically receives the 
data from the Satellite Data Handling System (SDFIS) Ingest Subsystem. SDFIS 
processes the raw data from the satellite which sometimes have to be decoded 
so CDFS II can use the meteorological information. 

b. Level 2 

Differentiating the cloud types is very important. In Level 2, there 
are three algorithms used to accomplish this task. The algorithms used help to 
capitalize on the strengths of each satellite system. The highest spatial 
resolution comes from DMPSP/OLS data. The data from the DMPS/OLS has a 
weakness with only providing one infrared channel and one visible channel (HQ 
AFWA/DNXM, 2005). AVHRR or NOAA data provides more band channels but 
has less resolution compared to the DMSP satellite image. The last type of 
satellite used is the Geostationary, i.e., GOES and Meteosat. This type of 
satellite provides more band channels compared to DMSP but it is stationary and 
has less resolution. The algorithm to determine cloud type operates at two 
different modes. The first mode is cloud detection, where the sensor only detects 
clouds. This mode provides the best satellite data possible with no bias in the 
data (HQ AFWA/DNXM, 2005). This process is used frequently in the WWMCA 
to detect the presence of clouds. The second mode is used in models, where 
cloud-free information is needed in the analysis. There is an over bias in the 
data because partial clouds (in pixel) are removed from the analysis. Both 
modes are used when the algorithm processes data once through (HQ 
AFWA/DNXM, 2005). 

c. Level 3 

Level 3, cloud layering and typing, provides important information 

so the data can be processed. A layering algorithm is used to process the data 
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vertically. The algorithm is used from the Long wavelength infrared (LWIR) 
channel. This information contains pixels, which have been assigned as cloud- 
filled from the Level 2 process. The information is used by the clustering 
algorithm which assigns each pixel a cloud layer. RTNEPH algorithm is used to 
determine the cloud type based on a few parameters. The information comes 
from the different visible channels and IR images. It is based on height of the 
cloud. The information contained in each box has cloud amount, time of the 
image, cloud base and height, and other important information (HQ 
AFWA/DNXM, 2005). 

d. Level 4 

Integration of the three gridded cloud analyses occurs at this level. 
This is the final process when the WWMCA is completed. The three types of 
data are from DMSP, NOAA/AVHRR, and geostationary satellites (HQ 
AFWA/DNXM, 2005). There might be different valid times for each gridded 
satellite analysis. All cloud analysis data and surface observation are combined 
into one analysis by using an integration algorithm. The final product is the 
Worldwide Merged Analysis. The integration process is based on accuracy of 
the data and timeliness. First, total cloud amount is integrated before integration 
of each layer. It is considered more dependable than the other types of cloud 
information. This is true because individual layer fraction as small sample size 
and possible error in cloud height. Each cell is processed independently of other 
cells. Rules are used to determine if one input is better than the other. If there 
were no new analysis, since the last analysis, then the previous information 
continues to be used. If there is a new analysis, then there is a check to the 
timeliness of the data. The most current data is used for the analysis. Multiple 
timely data requires a more thorough examination of the data. The data is 
scanned to see if it is cloud-free or cloud-filled. 100 or 0 percent is assigned to 
the cloud fraction based on the data being cloud-free or cloud-filled. 
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However, if the multiple temporal analyses exist that do not meet 
the cloud-free or cloud filled criteria, then a different approach has to be used. 
The data is classified by an error estimate and this is used to determine which 
data is used in the merging process. If this fails, then an optimum interpolation 
(01) algorithm determines which analysis will be used (HQ AFWA/DNXM, 2005). 
This algorithm is important, as the final step in selecting which data to 
incorporate. Figure 5 shows the maximum distance between cloud layer heights 
in merging the clouds. The 01 algorithm also weighs each analysis carefully to 
determine the most accurate data. Figure 6 displays a decision flow chart the 
algorithm follows when merging clouds. This decision is based on the estimate 
errors, and the information given to the CDFS II. Layer merging occurs once 
there is a total cloud fraction. Like the total cloud fraction a rule-based procedure 
is used in merging the cloud layers. Layer cloud parameters are set when there 
was only one suitable analysis based on time, clear skies, or if the analysis is 
determined to be the most accurate. If this is the case then the cloud parameters 
are set as the total cloud amount. If there are a 100 percent clouds in the 
analysis, or the 01 technique is used, then a different process occurs in merging 
the data (HQ AFWA/DNXM, 2005). It is difficult to compute layer amounts 
because of is the smaller amounts of data compared to the total amount of 
clouds. 

A more complex algorithm has to be used because of the vertical 
cloud coverage is resolved separately for each satellite data type. Therefore, a 
selection of which sensor is more accurate and a master template has to be 
assigned to the data. This master will be used for all of the combined data. An 
01 is used for matching cloud top temperature when the cloud amount is 
constantly changing. The integration algorithm is capable of using 
supplementary information. Once everything is processed, then the grid box is 
checked against this added data. The last data amount for each grid is then 
calculated. To compute the total amount of cloud coverage, it is total number of 
clouds divided by the total number of pixels (HQ AFWA/DNXM, 2005). Each 
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cloud layer is computed differently based on being a cold or warm layer. The 
coldest layer is similarly computed as the total cloud for the fractional layer 
amount. Layer amounts for warmer layers are computed differently. The total 
layer is adjusted based on under lapping assumptions. This assumption adjusts 
the layer vertically upward (Kiess and Cox, 1988). The assumption made is that 
a particular layer contains an equal amount of the whole sky as it would with no 
cloud amount. Naval Operational Global Atmospheric Prediction System 
(NOGAPS) is used to compute cloud top. RTNEPH technique is used to 
determine cloud base. The master analysis provides the cloud type and 
predominate cloud amount that is used in the cloud merger. Figure 7 and Figure 
8 shows cloud merger for single layers and for multiple layers. The time is 
assigned from the most recent analysis used in the merging process (HQ 
AFWA/DNXM, 2005). 


Description 

Units 

Cloud amount for each layer 

Percent 

Cloud base height for each 
layer 

Meters MSL 

Cloud top height for each 
layer 

Meters MSL 

Cloud type for each layer 

N/A 

Total Cloud Amount 

Percent 

Representative time 

Minutes 


Table 1. Worldwide Merge Analysis Products. (From: HQ AFWA/DNXM, 2005) 
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Diatanc* M a trie (m) (Mulmum layer taparett on to merge] 


Figure 5. Cloud Merge Default Distance Metric Response. (From: 

HQ AFWA/DNXM, 2005) 



Figure 6. Cloud Analysis Integration Functional Flow. (From: 

AFWA/DNXM, 2005) 
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Constellation 1 Constellation 2 Merged Result 
(Master) (Slave) 



7777777777777/7777777777777777777777777777 

Case 1 

Figure 7. 


Constellation 1 Constellation 2 Merged Result 
(Master) (Slave) 

O.T.O 


D 



777777777777777777777777777777777777777777 

Case 2 


Cloud Analysis Integration Example. (From: FIQ 
AFWA/DNXM, 2005) 


Constellation I Constellation 2 y . Merged Result 

(Master) (Slave) Comparison 



Figure 8. Multiple Layer Cloud Analysis Integration Example. (From: 

HQ AFWA/DNXM, 2005) 


C. OPEN SOURCE TOOLS 
1. MapServer 

MapServer is a tool that will be used to manipulate and render map 
images. It is used because it can create map images from a multitude of spatial 
data. Over 20 different vector data formats can be rendered by MapServer. 
Some examples of data formats are PostGis, shapefiles, ArcSDE and other 
vector formats. MapServer also can process other types of data formats, such as 

18 


















































bitmaps and GIFs (Kropla, 2005). A significant distinction should be made 
between vector data and raster data since each is used and stored differently. A 
vector representation of a geometrical object essentially consists of a list of the 
coordinates of the points that define the object. A raster object, on the other 
hand, consists of a string of values that represent the digital image. A vector 
object contains explicit spatial references by definition. A raster object, since it’s 
just an image, requires tags that allow it to be properly positioned, oriented, and 
scaled (Kropla, 2005). MapServer operates in two modes, which allows the user 
the flexibility to use various programs to access MapServer. The first mode, CGI 
mode, uses a web server as the primary user interface. The advantage to using 
this mode is that one can produce an application quickly. The second mode, 
MapScript, gives the user more tools to manipulate MapServer. In this mode, 
PHP, Python, or Perl can be used to access MapServer to create various 
applications (Kropla, 2005). MapServer reads a mapfile and text file, and 
transforms the information into a map. It gathers information from other 
templates which the user has listed in the HTML file. MapServer maps all of the 
features, and quickly sends the images from the tags to the web server. The 
web server then takes the data, and sends the information to the web browser. 
The web browser displays the map, and any layers added to the map by the 
user. The main benefit of using MapServer is that it can layer many rendered 
images on top of a map and quickly display these images (Kropla, 2005). 
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III. DATA AND METHODS 


A. INTRODUCTION 

This chapter shows the details of the data, and the methods used to 
extract information about the data. First, the observational data is gathered and 
stored in a spreadsheet. Then, the WWMCA data is gathered, and compared to 
the observational data. Chapter IV will show the verification of the WWMCA data 
compared to the synoptic weather observations. This graph will show the 
distribution of the data over a 16 day period for the 10 bases selected in this 
study. 

B. DATA 

1. Observational 

The synoptic weather observations were gathered from National Oceanic 
& Atmospheric Administration’s (NOAA) National Weather Service Aviation 
Weather Center, Aviation Digital Data Service (ADDS). The surface 
observations were requested daily from the ADDS website for 16 days, where 
hourly weather observations were saved. The cloud height and amount were 
tracked daily in a spreadsheet. Table 2 shows an example of the surface weather 
observations recorded for Tinker AFB, Oklahoma. The table shows observations 
for two days at Tinker AFB. The cloud amounts are in octas, and cloud height is 
in feet above the station (or AGL). For example, BKN060 is 5 to 7 octas for the 
cloud amount. The height is at 6,000 ft above Tinker AFB, where the surface 
weather observation was taken. 
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TIME 

Observed 

WWMCA 

Observed 


3-May-07 

3-May-07 

4-May-07 

OOZ 

SCT035 OVC075 


FEW030 SCT060 BKN250 

01Z 

OVC085 


BKN140 BKN250 

02Z 

BKN085 


SCT100 SCT150 BKN250 

03Z 

OVC027 


FEW050 SCT100 BKN250 

04Z 

SCT030 BKN090 OVC250 


FEW009 BKN050 BKN120 

05Z 

SCT012 OVC030 


OVC006 

06Z 

SCT006 BKN013 OVC022 


BKN008 

07Z 

OVC004 


OVC008 

08Z 

OVC004 


OVC007 

09Z 

W002 


OVC008 

10Z 

W002 


OVC007 

11Z 

OVC002 


OVC005 

12Z 

OVC002 


OVC003 

13Z 

W003 


OVC004 

14Z 

OVC002 


OVC004 

15Z 

OVC002 


OVC004 

16Z 

OVC003 


BKN008 

17Z 

OVC005 


BKN014 OVC018 

18Z 

BKN014 


BKN018 

19Z 

BKN030 


SCT022 

20Z 

BKN046 


SCT026 

21Z 

BKN048CB BKN250 


SCT029 

22Z 

SCT034CB BKN050 BKN250 
SCT030TCU BKN060 BKN160 


SCT029 

23Z 

BKN250 


SCT030 


*Automated Observation with no human 
augmentation 

Table 2. Surface Weather Observation for Tinker AFB, 

3 May 23, 2007 - 4 May 24, 2007. 

Table 3 shows the breakdown of the cloud amount. VV means the vertical 
visibility is totally obscured with 8/8 cloud coverage. Overcast skies (OVC), 
indicates total cloud coverage of 8/8. SKC or Clear skies means there are no 
clouds. 
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Table 9-2. Reportable Contractions for Sky Cover 


Reportable Contraction 

Meaning 

Summation Amount 
of Laver 

W 

Vertical Visibility 

oo 

CO 

SKCorCLR 1 

Clear 

0 

FEW 2 

Few 

oo 

1 

to 

CO 

SCT 

Scattered 

3/8 - 4/8 

BKN 

Broken 

5/8 -7/8 

ovc 

Overcast 

8/8 

1. The abbreviation C'LR shall be used at automated stations when no layers at or below 12,000 feet are 
reported: the abbreviation SKC shall be used at manual stations when no layers are reported. 

2. Any layer amount less than 18 is reported as FEW. 


Table 3. Reportable Contractions for Sky Cover. (From: U.S. Department of 
Commerce/National Oceanic and Atmospheric Administration, OCFM, 

2005) 

The WWMCA data, and synoptic weather observations were collected 
from 10 different U.S. Air Force Bases across the continental United States. The 
reason for choosing the United Stated is because of the large amount of data 
available, and the different climate regimes. The bases chosen were Dover AFB, 
Keesler AFB, Maxwell AFB, Nellis AFB, Offutt AFB, Patrick AFB, Scott AFB, 
Tinker AFB, Travis AFB, and Wright-Patterson AFB. The Pacific Northwest was 
not discussed in this study because of the various weather changes, which can 
occur. The Pacific Northwest may warrant a separate study. The study should 
also look at verification of cloud modeling to the synoptic observation. The 
military bases chosen offered 24-hours of continuous weather observations, 
which is very important for this study. There were six bases which used an 
automated weather sensor to observe for some part of the day. The automated 
weather sensor monitors the weather with or without a certified weather observer 
to augment the sensor. This is noted in this study and the results are broken into 
observer only and automated sensor only, because of the significant impact the 
automated data could have to the overall study. The automated cloud measuring 
equipment does not measure past 12,000ft above ground level (AGL). Thus, any 

cloud above 12,000ft will not get recorded in the observation, and will show up as 
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clear skies in the observation. WWMCA will show these data. This presented a 
problem in this study. It is difficult to know when there were clouds or no clouds 
in an observation generated by an automated sensor. The surface weather 
observations had to be extrapolated, and the comparisons made carefully 
between the observations and the WWMCA data. Another problem occurred 
when the surface observation had overcast conditions, and the model data 
showed obscured condition at this level. WWMCA often carried a cloud layer 
above the obscured layer. The observer or automated sensor does not record 
cloud layers above an obscured cloud layer. Therefore, it is difficult to say how 
accurate the cloud model was compared to the surface observation. 

Some assumptions had to be made in this study to verify the cloud model 
data. The first assumption made is that the surface observation is accurate. As 
mentioned before, this is difficult because some of the observations are 
monitored by an automated weather sensor at certain times of the day. Also, 
human weather observers have different observing skills, and different levels of 
experience. The observer might see the same cloud differently causing a 
different height to be recorded. The second assumption made in this study, if 
there is an overcast layer observed, and the WWMCA forecasted the same cloud 
height, cloud amount, and clouds above this obscured layer that this verified. 
The reasoning behind this is that the observer does not record cloud heights 
above the overcast layer, thus there could be multiple clouds above this 
obscured layer. 

2. World Wide Merge Cloud Analysis (WWMCA) 

The World Wide Merge Cloud Analysis was gathered from the Air Force 
Weather Agency (AFWA) daily. The development of the pseudo-surface 
observation program itself is beyond the scope of this thesis but the use of the 
program is what is important. The program allows the user to select locations to 
view by typing in the latitude and longitude, or the International Civil Aviation 
Organization (ICAO) name of the base. For example, the ICAO name for Tinker 
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AFB is KTIK. Once the location is identified, then the date and time of WWMCA 
is needed to access the information. The program outputs the location’s name 
and elevation height in feet and meters. The WWMCA data are in Mean Sea 
Level (MSL) in meters. The program converts the data into above ground level 
(AGL), the international standard for reporting clouds, and then converts the 
height into feet. This allows for an easier comparison to the observed data since 
the observing stations also reports the cloud height in feet. The program also 
lists the cloud type and cloud amount. An example of an output would be, KTIK 
131200Z (100)CU055. The KTIK is the location Tinker AFB, and the date and 
time in Zulu which follows the ICAO. The next text output is (100)CU055. This is 
the cloud height at 5,500ft AGL, and the total amount. The total amount in 
parenthesis is displayed as percentage, thus 100 percent is overcast at this 
layer. If there were no clouds forecasted at a particular date and time, then it will 
be displayed as SKC, or clear skies. There is some conversion made to 
compare the observed data to the model data. Table 4 shows the recorded 
surface weather observations and the WWMCA data for Tinker AFB, on May 3, 
2007. The WWMCA data and observations are for a 24 hour period. The Tower 
Cumulus (TCU) and Cumulus Nimbus (CB) are recorded but the type of cloud 
was not compared in the study. The WWMCA also classifies clouds into one of 
nine types (HQ AFWA/DNXM, 2005). It is difficult to compare the WWMCA cloud 
type to the surface weather observation because the observation does not show 
cloud type. It only shows cloud amount and height. It does show CB and TCU 
when the observer notes it at that particular layer. 
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TIME 

Observed 

WWMCA 


3-May-07 

3-May-07 


SCT035 OVC075 

(100)055 

01Z 

OVC085 

(100)083 

02Z 

BKN085 

(100)085 

03Z 

OVC027 

(100)222 (026)225 

04Z 

SCT030 BKN090 OVC250 

(100)214 (007)237 

05Z 

SCT012 OVC030 

(100)187 (081)263 

06Z 

SCT006 BKN013 OVC022 

(100)080 (100)212 (083)263 

07Z 

OVC004 

(100)103 (070)198 

08Z 

OVC004 

(100)130 (093)235 

09Z 

W002 

(026)094 


W002 

(100)189 (011)231 

11Z 

OVC002 

(100)254 

12Z 

OVC002 

(100)215 

13Z 

W003 

(100)221 

14Z 

OVC002 

(100)114 (015)203 

15Z 

OVC002 

(100)CB039 (079)263 

16Z 

OVC003 

(050)008 (100)201 

17Z 

OVC005 

(100)003 (037)087 

18Z 

BKN014 

(100)004 

19Z 

BKN030 

(096)014 


BKN046 

(047)027 (019)071 

21Z 

BKN048CB BKN250 

(047)030 (031)082 (004)182 


SCT034CB BKN050 BKN250 

(100)003 (044)114 


SCT030TCU BKN060 BKN160 

(097)031 (050)CB031 (049)235 

23Z 

BKN250 

(033)CB116 

Table 4. 

WWMCA Data and Surface Weather Observation for Tinker AFB, 


3 May 23, 2007. 


C. METHODS 

1. Comparison Between WWMCA and Surface Weather 
Observations 

The surface weather observation and WWMCA data were gathered over a 
16 day period. The data to be compared were the cloud height, and cloud 
amount because both data were provided by the WWMCA, and the surface 
weather observations. The comparison was for a 24 hour period starting at 00 
Zulu (Z) and ended at 23Z. Any missing data, or any automated data from the 
surface weather observation, or the missing WWMCA data were noted in the 
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final results. The complete surface observations and WWMCA data were 
recorded in the result. The verification approach taken was to look at certain 
characteristics of the observation independently. The synoptic weather 
observation and WWMCA data verified only if two conditions were met. First, the 
height of the WWMCA cloud had to be +/- 2000ft of the actual surface 
observation. Second, the cloud amount had to be the exact cloud amount of the 
observation. The grey area occurs when there were multiple cloud layers in the 
WWMCA data. The WWMCA data might get the first cloud layer correct but miss 
the other layers. 

There were numerous extrapolations made in assigning a layer as verified 
because of the multiple cloud layers. A color coding scheme was used to assign 
layers as verified or did not verified. The color green was assigned to a verified 
cloud layer. The color red was used when the cloud did not match. Also, if the 
cloud height was correct but the cloud amount was off then a color yellow was 
assigned to this layer. If the cloud amount was correct, but the cloud height was 
off then the color blue was used. If the data from either the WWMCA data, or the 
surface weather observation were missing, then the layer was not color coded. 
The comparisons were tracked in an excel spreadsheet for the 10 bases. 

The results from the 16 day comparison will be covered in Chapter IV. 
The next chapter will look at the total amount of observations recorded, and the 
breakdown of each base’s cloud data. The detail of the verification of the 
WWMCA data to the synoptic weather observation will be looked at carefully. 
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IV. RESULTS 


A. OVERVIEW 

The focus of this study is how the WWMCA data compared with the actual 
surface observations. First, the overall verification of the data will be examined 
to see any noticeable trends in the study. Second, the verification of the cloud 
height will be examined. The next area that will be verified will be the cloud 
amount. Finally, a comparison of all three areas will be shown in a graph for 
each Air Force base. 

B. VERIFICATION OF WWMCA DATA TO THE SURFACE WEATHER 

OBSERVATION 

The overall verification of the WWMCA cloud data using the surface 
weather observation was poor. The WWMCA data did well when there were no 
clouds, and clear skies were forecasted. It did not handle the increase of clouds, 
or any cloud layering. When there were numerous cloud layers it had a difficult 
time verifying. Thus, WWMCA data had a tougher time in regions of the United 
States where there were various cloud changes due to fronts, or upper level jets. 
The regions where the WWMCA handled the clouds well were in regions where 
the weather was easily predicted, and there were fewer major synoptic changes. 
The regions that faired well were Travis AFB, and Nellis AFB because of the 
overall quiet weather. 

1. Overall Verification of the Cloud Height, and Cloud Amount 
Between the WWMCA and Surface Observations 

Figure 9 shows results of the study from Nellis AFB. The study shows 

Nellis AFB had a 59% verification rate. This was not surprising since Nellis AFB 

is in a drier climate region. The results of Nellis AFB, shows the base with a 28% 

rate of not verifying. This was lower than some of the other bases where the rate 

of not verifying was much higher. The rest of the chart indicates a 5% rate with 
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the same cloud height and 8% rate where there was the same cloud amount. 
There were 367 total observations recorded for Nellis AFB. 


Nellis AFB, 3 May 07 -18 May 07 


□ Green - Same Cloud Height/ Cloud 
Coverage 

■ Red - No Similitary 

□ Yellow - Same Cloud Height (+/-2000ft) 

■ Blue - Same Cloud Coverage 


Figure 9. Total Observed results for Nellis AFB. 

Figure 10, Wright-Patterson AFB, had a lower percentage of verification 
as compared to Nellis AFB. It had 23% as verified and 42% as not verifying. 
The rest went to 24% rate, for the same cloud height, and 11% with the same 
cloud amount. During this time frame, Wright-Patterson AFB had more days of 
weather such as thunderstorms and rainshowers. This is a possible reason for 
the lower percentage of verification. 

Next, Offutt AFB had a higher percentage of verifying as compared to 
Wright-Patterson AFB. Figure 11 shows Offutt AFB with a 38% verification rate 
and a 33% of not verifying. Offutt AFB had a higher percentage of clouds at the 
same level with 26%. The total amount of observations, from Offutt AFB, was 
377, which a little higher then Wright-Patterson AFB, total observations 368. 
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WrightPatterson AFB, 3 May 07 -18 May 07 


■ 11% 



■ 42% 

Total Observations 368 


□ Green - Same Cloud Height/ Cloud 
Coverage 

■ Red - No Similitary 




□ Yellow - Same Cloud Height (+/-2000ft) 


■ Blue - Same Cloud Coverage 


Figure 10. Total Observed results for Wright-Patterson AFB. 


Since, the number of observations between Offutt AFB and Wright-Patterson 
AFB are a comparable amount. Then it is reasonable to say Offutt AFB did 
better verifying. 


Offutt AFB, 3 May 07 -18 May 07 


■ 3% 



■ 33% 

Total Observations 377 


□ Green - Same Cloud Height/ Cloud 
Coverage 

■ Red - No Similitary 

□ Yellow - Same Cloud Height (+/-2000ft) 

■ Blue - Same Cloud Coverage 


Figure 11. Total Observed results for Offutt AFB. 


Tinker AFB had a 1% better rate of verification as compared to Wright- 
Patterson AFB. However, it had a significant lower rate for not verifying with 
23%. This is significant since Tinker AFB is in the Midwest. Usually, the 
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Midwest has a lot of thunderstorms associated with frontal systems. Thus, there 
would be an expectation for a lower verification rate. But the study shows this is 
not the case. For some of the days, Tinker AFB reported lower clouds below 
1000 ft. WWMCA did well with verifying with the same low clouds. The study 
also shows Tinker AFB did well verifying for a longer period with low clouds. 
Figure 12 shows Tinker AFB having 40% rate with the same cloud height and the 
remaining 13% went to the same cloud amount. 


Tinker AFB, 3 May 07 -18 May 07 


□ 13 % 



Total Observations 376 


□ Green - Same Cloud Height/ Cloud 
Coverage 

■ Red - No Similitary 

□ Yellow - Same Cloud Height (+/-2000ft) 

■ Blue - Same Cloud Coverage 


Figure 12. Total Observed results for Tinker AFB. 


Figure 13 shows the combined results for the four Air Force bases 
mentioned previously. The overall verification for these bases was about 35% 
and 32% of not verifying. The same cloud height had about 24% and the same 
cloud coverage was lower with 9%. The total number of observations recorded 
for Nellis AFB, Offutt AFB, Tinker AFB, and Wright-Patterson AFB was 1488. 
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Nellis AFB, Offutt AFB Jinker AFB, and Wright-Patt AFB 
3 May 07 -18 May 07 


Figure 13. Total Observed results for Nellis AFB, Tinker AFB, and 

Wright-Patterson AFB. 

Figure 14 shows the total number of observations by an observer for 
Dover AFB, Keesler AFB, Maxwell AFB, Patrick AFB, Scott AFB, and Travis 
AFB. The total observations were 1166 over the 16 day period. The verification 
for this time was 27% and there was a 31% rate where the WWMCA did not 
verify. Figure 15 indicates 1105 automated observations taken during this study. 
There was a 42% verification rate for the six bases and a higher rate of misses 
with 49%. Once again, it is difficult to determine if there were clouds above 
12,000 ft, where the automated sensor cannot measure. Thus, the WWMCA 
could have verified at this height if the sensor had the capability to measure 
above the 12,000 ft threshold. This uncertainty should be noted even though the 
sensor is accurate and certified for operational use. 

Observations were separated into observer only and automated sensor 
only. Figure 16 shows the total number of observations for the six bases, which 
includes the observer only and automated sensor only. The combination of this 
shows a 36% rate for verifying and a 42% rate for misses. The trend for most of 
the bases was a high rate of misses, for the automated sensor, and a low rate, 
for observed only. 


□ Green - Same Cloud Height/ Cloud 
Coverage 

■ Red - No Similitary 

□ Yellow - Same Cloud Height (+/-2000ft) 

■ Blue - Same Cloud Coverage 


■ 9% 



■ 32% 

Total Observations 1488 
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Dover AFB, Keesler AFB, Maxwell AFB, Patrick AFB, Scott AFB, 
Travis AFB, 3 May 07-18 May 07 


□ 33% 



27% 


31% 

Total Observations 1166 


□ Green - Same Cloud Height/ Cloud 
Coverage 

■ Red - No Similitary 

□ Yellow - Same Cloud Height (+/-2000ft) 
n Blue - Same Cloud Coverage 


Figure 14. Total Observed results for Dover AFB, Keesler AFB, 
Maxwell AFB, Patrick AFB, Scott AFB, and Travis AFB. 


Dover AFB, Keesler AFB, Maxwell AFB, Patrick AFB, Scott AFB, 
Travis AFB, 3 May 07-18 May 07 



□ Green - Same Cloud Height/ Cloud 
Coverage 

■ Red - No Similitary 

□ Yellow - Same Cloud Height (+/-2000ft) 

□ Blue - Same Cloud Coverage 


Total Observations 1105 


Figure 15. Total Automated Sensor results for Dover AFB, Maxwell 
AFB, Patrick AFB, Scott AFB, and Travis AFB. 
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Dover AFB, Keesler AFB, Maxwell AFB, Patrick AFB, Scott AFB, 

Travis AFB, 3 May 07-18 May 07 

■ 6% 

□ Green - Same Cloud Height/ Cloud 
Coverage 

■ Red - No Similitary 

□ Yellow - Same Cloud Height (+/-2000ft) 

■ Blue - Same Cloud Coverage 

Figure 16. Total Observation results for Dover AFB, Keesler AFB, 
Maxwell AFB, Patrick AFB, Scott AFB, and Travis AFB. 

The Satellite image below, Figure 17, shows an infrared image, GOES 
East for May 17, 2007. The image was taken at 1615Z and shows a frontal 
system to the east and extensive cloudiness over the Ohio Valley. There is also 
a cloud system in the Texas and New Mexico area. Figure 18 shows a map of 
the continental United States with the ten bases on top of the map. The cloud 
depiction was created by MapServer from WWMCA raw data. It also shows the 
cloudiness over the eastern coast of the United States and over the Ohio Valley. 
There are clouds over the Texas and New Mexico area. Even though the 
computer generated cloud does not show the height of the clouds, it does show 
cloud amount. 

There is a definite benefit in using MapServer application to resolve the 
spatial and temporal scale of the WWMCA, and create a cloud images on a map 
background. This same map could be used to display other weather information 
such as 500 millibar heights or 850 millibar temperature. 



■ 42% 

Total Observations 2271 


35 









Figure 17. Goes East IR Satellite Image provided by NOAA showing 
the continental United States for 17 May 2007 at 1615Z. (From: 

NOAA, 2007) 


36 










Figure 18. Map created by MapServer with WWMCA data creating 
clouds over the continental United States for 17 May 2007. 
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V. SUMMARY AND FUTURE WORK 


A. SUMMARY 

The results from Chapter IV show a discrepancy in the verification with the 
drier climate regions and fair climate regions as compared to the more active 
weather regions. Nellis and Travis AFB had a higher fraction of verified 
observations compared to the other eight bases. Maxwell AFB had the highest 
percentage of poorly verifying with 44% from the observer only results. It was 
difficult to compare all the Air Force bases together because of the different ways 
of recording an observation (e.g., observer or automated) between the reporting 
stations. The results had to be separated in order to show a more accurate 
result on how the WWMCA verified. Overall, the WWMCA did not verify well with 
a verification of 27% and a miss rate of 32%. Therefore, the Air Force Weather 
Agency (AFWA) needs to look at further improving cloud model output. This 
study shows some of the shortcomings of WWMCA cloud model data and the 
potential benefits to AFWA if improvements are made to cloud model output. 
These verification methods may have to be refined in order to improve model 
performance. 


B. FUTURE WORK 

There should be continued study into verifying the WWMCA. The study 
should cover other regions such as Korea, the Middle East, and other significant 
areas of interest. There also should be a concentration in studying the Pacific 
Northwest and perhaps the New England area. Verification methods might have 
to be changed to fit a particular study. Other verification methods should be 
added to augment the lack of observations at a particular observation site. This 
could include satellite images or remote sensing images. Future studies should 
also focus on seasonal variations in the climate. This could make significant 
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improvements to the WWMCA model if any biases in the model could be found 
for a particular season. The duration of the study for future works might have to 
be increased to three to six months. Also, automating the synoptic observation 
retrieval process will be very beneficial for future studies. 
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